# squiggly
**Repository Path**: myking5200/squiggly
## Basic Information
- **Project Name**: squiggly
- **Description**: No description available
- **Primary Language**: Java
- **License**: BSD-3-Clause
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No
## Statistics
- **Stars**: 0
- **Forks**: 0
- **Created**: 2022-04-13
- **Last Updated**: 2022-04-13
## Categories & Tags
**Categories**: Uncategorized
**Tags**: None
## README
## Important Note
As of version 1.3.2, the preferred way to specify nested filters is to use square brackets intead of braces.
Preferred: `assignee[firstName]`
No longer Preferred but will still work: `assignee{firstName}`
The reason for this is that newer versions of Tomcat no longer allow braces to be specified on the url without being
escaped. Square brackets are still permitted in the url and it is preferred to make the syntax url friendly.
# Squiggly Filter For Jackson
## Contents
* [What is it?](#what-is-it)
* [Prerequisites](#prerequisites)
* [Installation](#installation)
* [General Usage](#general-usage)
* [Reference Object](#reference-object)
* [Top-Level Filters](#top-level-filters)
* [Nested Filters](#nested-filters)
* [Dot Syntax](#dot-syntax)
* [Regex Filters](#regex-filters)
* [Other Filters](#other-filters)
* [Resolving Conflicts](#resolving-conflicts)
* [Excluding Fields](#excluding-fields)
* [Property Views](#property-views)
* [More Examples](#more-examples)
* [Custom Integration](#custom-integration)
* [Changing the Defaults](#changing-the-defaults)
* [Metrics](#metrics)
* [Limitations](#limitations)
## What is it?
The Squiggly Filter is a [Jackson JSON](http://wiki.fasterxml.com/JacksonHome) PropertyFilter, which selects properties
of an object/list/map using a subset of the [Facebook Graph API filtering syntax](https://developers.facebook.com/docs/graph-api/using-graph-api/).
Probably the most common use of this library is to filter fields on the querystring like so:
```
?fields=id,reporter[firstName]
```
Integrating Squiggly into your webapp is covered in [Custom Integration](#custom-integration).
## Requirements
- Java 7+
- [ANTLR](http://www.antlr.org/)
- [Commons Lang 3](https://commons.apache.org/proper/commons-lang/)
- [Google Guava](https://github.com/google/guava)
- [Jackson JSON](http://wiki.fasterxml.com/JacksonHome) (version 2.6+)
## Installation
### Maven
```xml
com.github.bohnman
squiggly-filter-jackson
1.3.18
```
## General Usage
```java
ObjectMapper objectMapper = Squiggly.init(new ObjectMapper(), "assignee{firstName}");
Issue object = new Issue(); // replace this with your object/collection/map here
System.out.println(SquigglyUtils.stringify(objectMapper, object));
```
Alternatively, if you need more control over configuring the ObjectMapper, you can do it this way:
```java
String filterId = SquigglyPropertyFilter.FILTER_ID;
SquigglyPropertyFilter propertyFilter = new SquigglyPropertyFilter("assignee[firstName]"); // replace with your filter here
SimpleFilterProvider filterProvider = new SimpleFilterProvider().addFilter(filterId, propertyFilter);
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setFilterProvider(filterProvider);
objectMapper.addMixIn(Object.class, SquigglyPropertyFilterMixin.class);
Issue object = new Issue(); // replace this with your object/collection/map here
System.out.println(SquigglyUtils.stringify(objectMapper, object));
```
Also, you can generate a Plain Old Java Object (POJO) instead of a JSON String
```java
ObjectMapper objectMapper = Squiggly.init(new ObjectMapper(), "assignee{firstName}");
System.out.println(SquigglyUtils.objectify(objectMapper, object, Object.class));
```
For applying filter on Collection of Objects and for returning Collection of POJOs instead of JSON String
```java
List users = Arrays.asList(
new User("Peter", 12, "Dinklage"),
new User("Lena", 13, "Heady"));
String filter = "firstName,age";
ObjectMapper objectMapper = Squiggly.init(new ObjectMapper(), filter);
List filteredUsers = SquigglyUtils.listify(objectMapper, users, User.class);
// setify is also availble
```
## Reference Object
For the filtering examples, let's use an the example object of type Issue
```json
{
"id": "ISSUE-1",
"issueSummary": "Dragons Need Fed",
"issueDetails": "I need my dragons fed pronto.",
"reporter": {
"firstName": "Daenerys",
"lastName": "Targaryen"
},
"assignee": {
"firstName": "Jorah",
"lastName": "Mormont"
},
"actions": [
{
"id": null,
"type": "COMMENT",
"text": "I'm going to let Daario get this one.",
"user": {
"firstName": "Jorah",
"lastName": "Mormont"
}
},
{
"id": null,
"type": "CLOSE",
"text": "All set.",
"user": {
"firstName": "Daario",
"lastName": "Naharis"
}
}
],
"properties": {
"priority": "1",
"email": "motherofdragons@got.com"
}
}
```
## Top-Level Filters
### Select No Fields
```java
String filter = "";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {}
```
### Select Single Field
```java
filter = "id";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"id":"ISSUE-1"}
```
### Select Multiple Fields
```java
filter = "id,issueSummary"
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"id":"ISSUE-1", "issueSummary":"Dragons Need Fed"}
```
### Select Fields Using Wildcards
```java
filter = "issue*";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed", "issueDetails": "I need my dragons fed pronto."}
```
### Select All Fields
```java
filter = "**";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints the same json as our example object
```
### Select All Fields of object, but only base fields of associated objects (more on this later)
```java
filter = "*";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
/* prints the following:
{
"id": "ISSUE-1",
"issueSummary": "Dragons Need Fed",
"issueDetails": "I need my dragons fed pronto.",
"reporter": {
"firstName": "Daenerys",
"lastName": "Targaryen"
},
"assignee": {
"firstName": "Jorah",
"lastName": "Mormont"
},
"actions": [
{
"id": null,
"type": "COMMENT",
"text": "I'm going to let Daario get this one.."
},
{
"id": null,
"type": "CLOSE",
"text": "All set."
}
],
"properties": {
"priority": "1",
"email": "motherofdragons@got.com"
}
}
*/
```
## Nested Filters
### Select Single Nested Field
```java
String filter = "assignee[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"firstName":"Jorah"}}
```
### Select Multiple Nested Fields
```java
String filter = "actions[text,type]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"type":"COMMENT","text":"I'm going to let Daario get this one.."},{"type":"CLOSE","text":"All set."}]}
// NOTE: use can also use wildcards (e.g. actions{t*})
```
### Select Same Field From Different Nested Objects
```java
String filter = "(assignee,reporter)[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"reporter":{"firstName":"Daenerys"},"assignee":{"firstName":"Jorah"}}
```
### Select Deeply Nested Field
```java
String filter = "actions[user[lastName]]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"user":{"lastName":"Mormont"}},{"user":{"lastName":"Naharis"}}]}
```
## Dot Syntax
As an alternative to using the braces syntax for nested filter, you can use the dot syntax
```java
String filter = "assignee.firstName";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"firstName":"Jorah"}}
```
You can exclude fields using the dot syntax. Note that the exclusion applies to the last field.
```java
String filter = "-assignee.firstName";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"assignee":{"lastName":"Mormont"}}
```
You can also combine the dot syntax with the nested syntax.
```java
String filter = "actions.user[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"actions":[{"user":{"firstName":"Jorah"}},{"user":{"firstName":"Daario"}}]}
```
One limitation is that you cannot use the | syntax with the dot syntax
```java
String filter = "(actions.user,assignee)[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// throws exception
```
## Regex Filters
In addition to using wildcards, you can also use regular expressions.
Here's an example:
```java
String filter = "~iss[a-z]e.*~";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed","issueDetails":"I need my dragons fed pronto."}
```
Notice the tildes mark the begin and of the regex pattern.
You can also specifiy a case insensitive match.
```java
String filter = "~iss[a-z]esumm.*~i";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed"}
```
Why use tildes and not forward slashes for regular expressions? Tildes are query string friendly and forward slashes
are not.
However, you may use forward slashes if you like.
```java
String filter = "/iss[a-z]esumm.*/i";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints {"issueSummary":"Dragons Need Fed"}
```
## Other Filters
### Selecting from Maps
Selecting from maps is the same as selecting from objects. Instead of selecting from fields, you are selecting
from keys. The main downside of selecting from maps that their matches are unable to be cached.
```java
Map map = new HashMap<>();
map.put("foo", "bar");
map.put("bear", "baz");
String filter = "foo";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, map));
// prints {"foo":"bar"}
```
### Selecting from Collections (Lists/Arrays/Etc).
Selecting from collection just assumes the top-level objects are the elements in the collection, not the collection
itself.
```java
List list = Arrays.asList(
new User("Peter", "Dinklage"),
new User("Lena", "Heady")
);
String filter = "firstName";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, list));
// prints [{"firstName":"Peter"}, {"firstName":"Lena"}]
```
## Resolving Conflicts
When a filter includes two criteria that match the same field, the one that is more specific wins.
For example, if the filter is "**,reporter[firstName]", then all fields will be excluded. However, the reporter field
will only include the firstName field.
Specificity is determined using the following logic:
- an exact name is the most specific
- a ** is the least specific
- a * is the second to least specific
- otherwise, the number of non-wildcard characters is counted, the higher the number, the more specific
- if two filters have the same specificity, the latter one is chosen
## Excluding Fields
In order to exclude fields, you need to prefix the field name with a minus sign (-).
Let's look at an example
```java
String filter = "-reporter";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints everything except the reporter field
```
Here's an example excluding a nested field:
```java
String filter = "**,reporter[-firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// prints everything, the reporter object will only have the firstName excluded
```
NOTE: Excluded fields can't have nested filters
```java
String filter = "**,-reporter[firstName]";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, issue));
// throws an exception
```
## Property Views
In addition to selecting fields by name, you can assign a name to a group of fields. This is called a property view.
### Reference Objects
Let's use these reference objects for the examples.
```java
@Target(FIELD)
@Retention(RUNTIME)
@Documented
@PropertyView({"super"})
public @interface SuperView {
}
class Address {
String line1 = "55 Hollywood Blvd.";
String line2 = "";
String city = "Hollywood";
String state = "CA";
@SuperView
double lat;
@SuperView
double lon;
}
class User {
String firstName = "Peter";
String lastName = "Dinklage";
@PropertyView("secret")
String phone = "555-555-1212";
@SuperView
Address address;
}
```
### The Base View
If nothing is annotated on a field, it is assumed to belong to the "base" view. There is @BaseView convenience
annotation, but it's not needed. See [Changing Defaults](#changing-the-defaults) to alter this behavior.
In the case of User, fields firstName and lastName belong to the "base" view.
```java
String filter = "base";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage"}
```
### Using the @PropertyView Annotation
If you look at the phone field of the User class, you'll notice the `@PropertyView("secret")` annotation on the phone
field. This indicates that the phone field belongs to the "secret" view.
```java
String filter = "secret";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage", "phone":"555-555-1212"}
```
**Wait a minute!** Why was the firstName and lastName field included? Even though we specified a certain view, the
base fields are always included. See [Changing Defaults](#changing-the-defaults) to alter this behavior.
Note that you can also specifiy multiple views in the annotation - `@PropertyView({"one", "two", "three"}))`
You can also specify `@PropertyView` on getters and setters.
### Using a Derived Annotation
If you look at the address field of the User class, you'll notice the @SuperView annotation. Looking at the @SuperView
declaration, you'll notice it is annotated with a @PropertyView("super"). This is how you create a derived annotation.
Let's try it out.
```java
String filter = "super";
ObjectMapper mapper = Squiggly.init(mapper, filter);
System.out.println(SquigglyUtils.stringify(mapper, user));
// prints {"firstName":"Peter","lastName","Dinklage", "address":{"line1":"55 Hollywood Blvd.","line2":"","city":"Hollywood","state":"CA"}}
```
**Wait another minute!** The Address class has @SuperView annotations as well. Why weren't they include? Well, the
view only applies to the current level. In order to get the super views of the address, you would have to specifiy a
filter "super[super]". See [Changing Defaults](#changing-the-defaults) to alter this behavior.
## More Examples
There are more examples in the test directory.
## Custom Integration
Imagine you are building a webapp where you want to specify the fields on the querystring.
E.g. `/some/path?fields=a,b{c} ``
You'll notice in all of our examples, we passed in a filter expression that never changes. This doesn't
work well for the case of specifying filters on a querystring.
Enter the SquigglyContextProvider. This interface allows you to customize how to retrieve the fields.
### The RequestSquigglyContextProvider
All servlet-based integrations use the RequestSquigglyContextProvider, which has the general initialization in the form of:
```java
Squiggly.init(objectMapper, new RequestSquigglyContextProvider());
```
### Automatically wrapping fields with an outer filter
Let's say you have the following class called Page that looks like this:
```java
public class Page {
private final int pageNumber;
private final int pageSize;
private final List items;
public ListResponse(List items, int pageNumber, int pageSize) {
this.items = checkNotNull(items);
this.pageNumber = pageNumber;
this.pageSize = pageSize;
}
public List getItems() {
return items;
}
public int getPageNumber() {
return pageNumber;
}
public int getPageSize() {
return pageSize;
}
}
```
Let's say you have an endpoint called /issues that looks like this:
```java
public Page findIssues(String query, int pageNumber, int pageSize) {
List issues = issueService.findIssues(query, pageNumber, pageSize);
return new Page(issues, pageNumber, pageSize);
}
```
In order to get specify the issue property, you now have to wrap all filters with items[] like so:
```
GET /issues?fields=items{id}&query=some-query&&pageNumber=1&pageSize=10
```
This is kind of annoying. Fortunately, we can avoid this inconvenience by using a hook method in RequestSquigglyContextProvider.
Here's how it would look:
```java
Squiggly.init(objectMapper, new RequestSquigglyContextProvider() {
@Override
protected String customizeFilter(String filter, HttpServletRequest request, Class beanClass) {
if (filter != null && Page.class.isAssignableFrom(beanClass)) {
filter = "items[" + filter + "]";
}
return filter;
}
});
```
Now you can do this:
```
GET /issues?fields=id&query=some-query&&pageNumber=1&pageSize=10
```
### Generic Servlet Webapp
You can find an example of using Squiggly Filter in a webapp under the [examples/servlet](examples/servlet) directory.
### Spring Boot Web Application
You can find an example of using Squiggly Filter in Spring Boot under the [examples/spring-boot](examples/spring-boot) directory.
### Dropwizard
You can find an example of using Squiggly Filter in Dropwizard under the [examples/dropwizard](examples/dropwizard) directory.
## Changing Defaults
You have the ability to customize Squiggly by creating a file called squiggly.properties in the root of the classpath.
### Cache Config
The following properties are used to control various caches in Squiggly Filter. Internally, these properties get
converted to a Guava
[CacheBuilderSpec](https://google.github.io/guava/releases/19.0/api/docs/index.html?com/google/common/cache/CacheBuilderSpec.html).
Please refer to to the documentation to see all the values that are available.
- parser.nodeCache.spec=maximumSize=10000
- filter.pathCache.spec=maximumSize=10000
- property.descriptorCache.spec=<empty>
### Enable/Disable adding non-annotated fields to the "base" view
- property.addNonAnnotatedFieldsToBaseView=true
### Enable/Disable inclusion of base fields for nested objects
- filter.implicitlyIncludeBaseFields=true
### Enable/Disable inclusion of base fields when a view is specified
- filter.implicitlyIncludeBaseFieldsInView=true
When set to false, base fields are not included when specifying a view
### Enable/Disable View Propagation to Nested Filters
- filter.propagateViewToNestedFilters=false
When set to true, views are propagated to nested filters
## Getting Config Info
Squiggly Filter provides 2 methods to get information about configuration.
`SquigglyConfig.asMap()` will return a map of the merged config that looks like the following:
```json
{
"filter.implicitlyIncludeBaseFields": "true",
"filter.implicitlyIncludeBaseFieldsInView": "true",
"filter.pathCache.spec": "maximumSize=10000",
"filter.propagateViewToNestedFilters": "false",
"parser.nodeCache.spec": "maximumSize=10000",
"property.addNonAnnotatedFieldsToBaseView": "true",
"property.descriptorCache.spec": ""
}
```
`SquigglyConfig.asSourceMap()` will return a map of the config keys and paths where the entry was retrieved that looks
like the following:
```json
{
"filter.implicitlyIncludeBaseFields": "file:/path/one/squiggly.default.properties",
"filter.implicitlyIncludeBaseFieldsInView": "file:/path/one/squiggly.default.properties",
"filter.pathCache.spec": "file:/path/one/squiggly.default.properties",
"filter.propagateViewToNestedFilters": "file:/path/one/squiggly.default.properties",
"parser.nodeCache.spec": "file:/path/two/squiggly.properties",
"property.addNonAnnotatedFieldsToBaseView": "file:/path/two/squiggly.properties",
"property.descriptorCache.spec": "file:/path/two/squiggly.properties"
}
```
## Metrics
Squiggly Filter provides an API for obtaining various metrics about the library, such as cache statistics. This allows
users to monitor and adjust configuration as needed.
To use the metrics, you can do something the like following:
```java
Map metrics = SquigglyMetrics.asMap();
System.out.println(SquigglyUtils.stringify(new ObjectMapper(), metrics));
```
This will print the following:
```json
{
"squiggly.filter.pathCache.averageLoadPenalty": 0,
"squiggly.filter.pathCache.evictionCount": 0,
"squiggly.filter.pathCache.hitCount": 0,
"squiggly.filter.pathCache.hitRate": 1,
"squiggly.filter.pathCache.loadExceptionCount": 0,
"squiggly.filter.pathCache.loadExceptionRate": 0,
"squiggly.filter.pathCache.loadSuccessCount": 0,
"squiggly.filter.pathCache.missCount": 0,
"squiggly.filter.pathCache.missRate": 0,
"squiggly.filter.pathCache.requestCount": 0,
"squiggly.filter.pathCache.totalLoadTime": 0,
"squiggly.parser.nodeCache.averageLoadPenalty": 0,
"squiggly.parser.nodeCache.evictionCount": 0,
"squiggly.parser.nodeCache.hitCount": 0,
"squiggly.parser.nodeCache.hitRate": 1,
"squiggly.parser.nodeCache.loadExceptionCount": 0,
"squiggly.parser.nodeCache.loadExceptionRate": 0,
"squiggly.parser.nodeCache.loadSuccessCount": 0,
"squiggly.parser.nodeCache.missCount": 0,
"squiggly.parser.nodeCache.missRate": 0,
"squiggly.parser.nodeCache.requestCount": 0,
"squiggly.parser.nodeCache.totalLoadTime": 0,
"squiggly.property.descriptorCache.averageLoadPenalty": 0,
"squiggly.property.descriptorCache.evictionCount": 0,
"squiggly.property.descriptorCache.hitCount": 0,
"squiggly.property.descriptorCache.hitRate": 1,
"squiggly.property.descriptorCache.loadExceptionCount": 0,
"squiggly.property.descriptorCache.loadExceptionRate": 0,
"squiggly.property.descriptorCache.loadSuccessCount": 0,
"squiggly.property.descriptorCache.missCount": 0,
"squiggly.property.descriptorCache.missRate": 0,
"squiggly.property.descriptorCache.requestCount": 0,
"squiggly.property.descriptorCache.totalLoadTime": 0
}
```
## Limitations
### Using Serializers
If you use a custom serializer and write to a JsonGenerator directly, you will completely bypass Squiggly.
For example:
```java
// Doesn't work with Squiggly
public class TestSerializer extends JsonSerializer {
@Override
public void serialize(TestObject value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
jgen.writeStartObject();
jgen.writeStringField("a", value.getA());
jgen.writeStringField("c", value.getC());
jgen.writeEndObject();
}
}
```
Instead, you need to use the SerializerProvider, which will invoke Squiggly. The above serializer can be rewritten as:
```java
// Works with Squiggly
public class TestSerializer extends JsonSerializer {
@Override
public void serialize(TestObject value, JsonGenerator jgen, SerializerProvider provider) throws IOException {
Map map = new HashMap<>();
map.put("a", value.getA());
map.put("c", value.getC());
provider.defaultSerializeValue(map, jgen);
}
}
```