Protobuf - Compound Data Types



There are two more compound data types which may be useful for complicated use cases. They are "OneOf" and "Any". In this chapter, we will see how to use these two data types of Protobuf.

OneOf

We pass a few parameters to this OneOf data type and Protobuf ensures that only one of them is set. If we set one of them and try to set the other one, the first attribute gets reset. Let's us understand this via an example.

Continuing with our theater example, say, we have an API which is used to fetch the count of available employees. The value returned from this API is then set to 'count' tag in the following file. But if that API errors out, we can't really 'count', instead we attach the error log.

Ideally, we will always have one of them set, i.e., either the call is successful and we get the count OR the count calculation fails and we get the error message.

Following is the syntax that we need to have to instruct Protobuf that we will be creating an OneOf attribute −

syntax = "proto3";
package theater;
option java_package = "com.tutorialspoint.theater";

message Theater {
   string name = 1;
   string address = 2;
  
   repeated google.protobuf.Any peopleInside = 3;
  
   oneof availableEmployees {
      int32 count = 4;
      string errorLog = 5;
   }
}

Now our class/message contains an OneOf attribute, i.e., information about the available employees.

To use Protobuf, we will have to use protoc binary to create the required classes from this ".proto" file. Let us see how to do that −

protoc  --java_out=java/src/main/java proto_files\theater_advanced.proto

The above command should create the required files and now we can use it in our Java code. First, we will create a writer to write the theater information −

package com.tutorialspoint.theater;

import java.util.List;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import com.google.protobuf.Any;
import com.tutorialspoint.theater.TheaterAdvanced.Employee;
import com.tutorialspoint.theater.TheaterAdvanced.Viewer;
import com.tutorialspoint.theater.TheaterAdvanced.Theater;

public class TheaterWriterComplex{
   public static void main(String[] args) throws IOException {
      List<Any> people = new ArrayList<>();
      people.add(Any.pack(Employee.newBuilder().setName("John").build()));
      people.add(Any.pack(Viewer.newBuilder().setName("Jane").setAge(30).build()));
      people.add(Any.pack(Employee.newBuilder().setName("Simon").build()));
      people.add(Any.pack(Viewer.newBuilder().setName("Janice").setAge(25).build()));
	    
      Theater theater = Theater.newBuilder()
         .setName("SilverScreen")
         .addAllPeopleInside(people)
         .build();
		
      String filename = "theater_protobuf_output_silver";
      System.out.println("Saving theater information to file: " + filename);
		
      try(FileOutputStream output = new FileOutputStream(filename)){
         theater.writeTo(output);
      }
	    
      System.out.println("Saved theater information with following data to disk: \n" + theater);    
   }
}	

Next, we will have a reader to read the theater information −

package com.tutorialspoint.theater;

import java.io.FileInputStream;
import java.io.IOException;
import com.google.protobuf.Any;
import com.tutorialspoint.theater.TheaterAdvanced.Theater;
import com.tutorialspoint.theater.TheaterAdvanced.Theater.AvailableEmployeesCase;
import com.tutorialspoint.theater.TheaterAdvanced.Theater.Builder;
import com.tutorialspoint.theater.TheaterAdvanced.Viewer;
import com.tutorialspoint.theater.TheaterAdvanced.Employee;

public class TheaterReaderComplex{
   public static void main(String[] args) throws IOException {
	    
      Builder theaterBuilder = Theater.newBuilder();

      String filename = "theater_protobuf_output_silver";
      System.out.println("Reading from file " + filename);
        
      try(FileInputStream input = new FileInputStream(filename)) {
         Theater theater = theaterBuilder.mergeFrom(input).build();
         System.out.println("Name:" + theater.getName() + "\n");
            
         for (Any anyPeople : theater.getPeopleInsideList()) {
            if(anyPeople.is(Employee.class)) {
               Employee employee = anyPeople.unpack(Employee.class);
               System.out.println("Employee:" + employee + "\n");
            }
                
            if(anyPeople.is(Viewer.class)) {
               Viewer viewer = anyPeople.unpack(Viewer.class);
               System.out.println("Viewer:" + viewer + "\n");
            }
         }
      }
   }
}	

Now, post compilation, let us execute the writer first −

> java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterWriter

Saving theater information to file: theater_protobuf_output_silver
Saved theater information with following data to disk:
name: "SilverScreen"
peopleInside {
   type_url: "type.googleapis.com/theater.Employee"
   value: "\n\004John"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Viewer"
   value: "\n\004Jane\020\036"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Employee"
   value: "\n\005Simon"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Viewer"
   value: "\n\006Janice\020\031"
}

Now, let us execute the reader to read from the same file −

java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterReader

Reading from file theater_protobuf_output_silver
Name:SilverScreen

Employee:name: "John"

Viewer:name: "Jane"
age: 30

Employee:name: "Simon"

Viewer:name: "Janice"
age: 25

So, as we see, in the list, we are able to figure out the Any type and find the respective underlying datatype employee/viewer. Let us now look at defaults and AnyOf.

Any

The next data type that can be of use for complicated uses cases is Any. We can pass any type/message/class to this data type and Protobuf would not complain. Let us understand this via an example.

Continuing with the theater example, say, we want to track people inside the theater. Some of them could be employees and others could be viewers. But ultimately they are people, so we will pass them in a single list which would contain both the types.

Following is the syntax that we need to have to instruct Protobuf that we will be creating a list −

syntax = "proto3";
package theater;
option java_package = "com.tutorialspoint.theater";

import "google/protobuf/any.proto";

message Theater {
   string name = 1;
   string address = 2;
  
   repeated google.protobuf.Any peopleInside = 3;
}
message Employee{
   string name = 1;
   string address = 2;
}
message Viewer{
   string name = 1;
   int32 age = 2;
   string sex = 3;
}

Now our class/message contains an Any attribute 'peopleInside' list along with Viewer and Employee class, i.e., information about the people inside theater. Let us see this in action.

To use Protobuf, we will now have to use protoc binary to create the required classes from this ".proto" file. Let us see how to do that −

protoc  --java_out=java/src/main/java proto_files\theater_advanced.proto

The above command should create the required files and now we can use it in our Java code. First, we will create a writer to write the theater information −

package com.tutorialspoint.theater;

import java.util.List;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;

import com.google.protobuf.Any;
import com.tutorialspoint.theater.TheaterAdvanced.Employee;
import com.tutorialspoint.theater.TheaterAdvanced.Viewer;
import com.tutorialspoint.theater.TheaterAdvanced.Theater;

public class TheaterWriter{
   public static void main(String[] args) throws IOException {
	    
   List<Any> people = new ArrayList<>();
   people.add(Any.pack(Employee.newBuilder().setName("John").build()));
   people.add(Any.pack(Viewer.newBuilder().setName("Jane").setAge(30).build()));
   people.add(Any.pack(Employee.newBuilder().setName("Simon").build()));
   people.add(Any.pack(Viewer.newBuilder().setName("Janice").setAge(25).build()));
	    
   Theater theater = Theater.newBuilder()
      .setName("SilverScreen")
      .addAllPeopleInside(people)
      .build();
		
      String filename = "theater_protobuf_output_silver";
      System.out.println("Saving theater information to file: " + filename);
		
      try(FileOutputStream output = new FileOutputStream(filename)){
         theater.writeTo(output);
      }
	    
      System.out.println("Saved theater information with following data to disk: \n" + theater);
   }
}

Next, we will have a reader to read the theater information −

package com.tutorialspoint.theater;

import java.io.FileInputStream;
import java.io.IOException;
import com.google.protobuf.Any;
import com.tutorialspoint.theater.TheaterAdvanced.Theater;
import com.tutorialspoint.theater.TheaterAdvanced.Theater.AvailableEmployeesCase;
import com.tutorialspoint.theater.TheaterAdvanced.Theater.Builder;
import com.tutorialspoint.theater.TheaterAdvanced.Viewer;
import com.tutorialspoint.theater.TheaterAdvanced.Employee;

public class TheaterReaderComplex{
   public static void main(String[] args) throws IOException {
	   Builder theaterBuilder = Theater.newBuilder();

      String filename = "theater_protobuf_output_silver";
      System.out.println("Reading from file " + filename);
        
      try(FileInputStream input = new FileInputStream(filename)) {
         Theater theater = theaterBuilder.mergeFrom(input).build();
            
         System.out.println("Name:" + theater.getName() + "\n");
            
         for (Any anyPeople : theater.getPeopleInsideList()) {
            if(anyPeople.is(Employee.class)) {
               Employee employee = anyPeople.unpack(Employee.class);
               System.out.println("Employee:" + employee + "\n");
            }
                
            if(anyPeople.is(Viewer.class)) {
               Viewer viewer = anyPeople.unpack(Viewer.class);
               System.out.println("Viewer:" + viewer + "\n");
            }
         }
      }
   }
}	

Now, post compilation, let us execute the writer first −

> java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterWriter

Saving theater information to file: theater_protobuf_output_silver
Saved theater information with following data to disk:
name: "SilverScreen"
peopleInside {
   type_url: "type.googleapis.com/theater.Employee"
   value: "\n\004John"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Viewer"
   value: "\n\004Jane\020\036"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Employee"
   value: "\n\005Simon"
}
peopleInside {
   type_url: "type.googleapis.com/theater.Viewer"
   value: "\n\006Janice\020\031"
}

Note − There are two points to note −

  • In case of Any, Protobuf packs/serializes the contents inside any tag to bytes and then stores it as 'value'. Basically, that allows us to send any message type with this 'Any' tag.

  • We also see "type.googleapis.com/theater.Viewer" and "type.googleapis.com/theater.Employee". This is used by Protobuf to save the type of object along with the data as the type of data in the Any data type can vary.

Now let us execute the reader to read from the same file −

java -cp .\target\protobuf-tutorial-1.0.jar com.tutorialspoint.theater.TheaterReader

Reading from file theater_protobuf_output_silver
Name:SilverScreen

Employee:name: "John"

Viewer:name: "Jane"
age: 30

Employee:name: "Simon"

Viewer:name: "Janice"
age: 25

So, as we see, our reader code is successfully able to differentiate between Employee and the Viewer, even though they come in the same array.

Advertisements